refactor: split oversized files into focused modules (ALIEN-291)#133
refactor: split oversized files into focused modules (ALIEN-291)#133lilienblum wants to merge 31 commits into
Conversation
Break the 7,000-line worker/gcp.rs into a directory module so each concern can be read and reviewed in isolation: - gcp/mod.rs: controller struct, compute-operation tracking, and the #[controller] handler impl (kept whole for macro state generation) - gcp/support.rs: naming helpers, error classifiers, heartbeat emission, and GcsNotificationTracker - gcp/helpers.rs: the plain helper-method impl block - gcp/tests.rs: the controller test module Pure code motion: moved items are bumped to pub(super) where needed; external paths are unchanged via the existing worker::gcp re-exports.
Greptile SummaryThis PR splits the monolithic
Confidence Score: 5/5Pure code-motion refactor with no logic changes; the proc-macro input in mod.rs is byte-for-byte identical and all 478 tests pass. Every changed file is either a new module boundary file containing code moved verbatim from the original gcp.rs, or the slimmed-down mod.rs that retains the macro-processed handler block unchanged. The only non-motion deltas are the 14 pub(super) visibility bumps (all internal) and two rustfmt reflows from the dedent and a signature wrap — none affecting behavior. The re-export of GcsNotificationTracker correctly preserves the external API path. No files require special attention; the split is mechanical and the verification evidence (cargo check, nextest, byte-level diff) is thorough.
|
| Filename | Overview |
|---|---|
| crates/alien-infra/src/worker/gcp/mod.rs | Root module retaining the #[controller] struct and its handler impl block; declares helpers/support/tests sub-modules and re-exports GcsNotificationTracker via pub use support::GcsNotificationTracker. |
| crates/alien-infra/src/worker/gcp/helpers.rs | Extracted helper impl block with 14 methods bumped to pub(super); imports support items via use super::support::* and conditionally imports GcpWorkerState under test-utils feature. |
| crates/alien-infra/src/worker/gcp/support.rs | Free functions, consts, and GcsNotificationTracker type; all items are pub(super) except GcsNotificationTracker which is pub (re-exported from mod.rs) to preserve the external API path. |
| crates/alien-infra/src/worker/gcp/tests.rs | Test module moved out of the inline mod block with mandatory dedent; imports are adjusted to use super:: paths for support utilities, no logic changes. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A["worker/mod.rs\n(crate re-exports)"]
B["gcp/mod.rs\n(#[controller] struct\n+ handler impl block\n3,985 lines)"]
C["gcp/helpers.rs\n(helper impl block\n14 pub(super) methods\n1,422 lines)"]
D["gcp/support.rs\n(free fns, consts, types\n248 lines)"]
E["gcp/tests.rs\n(#[cfg(test)]\n1,441 lines)"]
A --> B
B -->|"mod helpers;"| C
B -->|"mod support;\nuse support::*;\npub use support::GcsNotificationTracker"| D
B -->|"#[cfg(test)] mod tests;"| E
C -->|"use super::support::*"| D
E -->|"use super::{ get_cloudrun_service_name, ... }"| D
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
A["worker/mod.rs\n(crate re-exports)"]
B["gcp/mod.rs\n(#[controller] struct\n+ handler impl block\n3,985 lines)"]
C["gcp/helpers.rs\n(helper impl block\n14 pub(super) methods\n1,422 lines)"]
D["gcp/support.rs\n(free fns, consts, types\n248 lines)"]
E["gcp/tests.rs\n(#[cfg(test)]\n1,441 lines)"]
A --> B
B -->|"mod helpers;"| C
B -->|"mod support;\nuse support::*;\npub use support::GcsNotificationTracker"| D
B -->|"#[cfg(test)] mod tests;"| E
C -->|"use super::support::*"| D
E -->|"use super::{ get_cloudrun_service_name, ... }"| D
Reviews (1): Last reviewed commit: "refactor: split gcp worker controller in..." | Re-trigger Greptile
Pure git mv of worker/azure.rs to worker/azure/mod.rs with no content changes, so rename detection and git log --follow stay intact ahead of the module carve-out.
Break the 6,500-line worker/azure.rs into a directory module so each concern can be read and reviewed in isolation: - azure/mod.rs: controller struct, environment-wake retry tracking, and the #[controller] handler impl (kept whole for macro state generation) - azure/support.rs: naming helpers, wait/delay helpers, error classifiers, heartbeat emission, and AzureStorageTriggerInfrastructure - azure/helpers.rs: the plain helper-method impl block - azure/tests.rs: the controller test module Pure code motion: moved items are bumped to pub(super) where needed; external paths are unchanged via the existing worker::azure re-exports.
Break the 5,000-line worker/aws.rs into a directory module so each concern can be read and reviewed in isolation: - aws/mod.rs: controller struct and the #[controller] handler impl (kept whole for macro state generation) - aws/support.rs: naming helpers, error classifiers, heartbeat emission, and the load balancer state types - aws/helpers.rs: the plain helper-method impl blocks - aws/tests.rs: the controller test module Pure code motion: moved items are bumped to pub(super) where needed; external paths are unchanged via the existing worker::aws re-exports.
Break the 5,700-line gcp/compute.rs into a directory module so each concern can be read and reviewed in isolation: - compute/mod.rs: module doc, ComputeServiceConfig, and re-exports - compute/api.rs: the ComputeApi trait (automock still generates MockComputeApi here, re-exported unchanged) - compute/client.rs: ComputeClient and its ComputeApi impl - compute/types/: the serde data structures, split along the existing section banners (operations, network, load_balancer, instance) - compute/tests.rs: the PSC serde test module Pure code motion: no items were renamed or reordered; external paths are unchanged via glob re-exports from compute/mod.rs. Note in gcp/AGENTS.md that compute/ is the directory-module variant of the cloudrun.rs pattern.
Break the 4,800-line lib.rs into concern modules so each area can be read and reviewed in isolation: - lib.rs: build_stack orchestration, build_resource, and build_target_to_file - push.rs: push_stack, push targets, OCI tarball selection, and image index assembly - cache.rs: artifact cache keys, source hashing, cargo metadata, and cached artifact lookup - base_images.rs: base image resolution, entrypoint/cmd handling, and pull/retry classification - tests.rs: the unit test module, still gated by #[cfg(test)] Pure code motion: moved items are bumped to pub(crate); the public API is unchanged via the root pub use of push_stack.
Break the 4,647-line generator.rs into a generator/ directory so each concern can be read and reviewed in isolation: - mod.rs: the options types, the generate_helm_chart and render_manager_fetch_values entrypoints, and shared JSON/YAML/name helpers - operator.rs: generate_operator_manifest and the operator document builders - values.rs: ChartAnalysis, values.yaml assembly, and cloud-identity mapping - schema.rs: values.schema.json - templates.rs: the Helm template (.tpl) bodies - examples.rs: the per-target values examples and README - tests.rs: the unit test module, still gated by #[cfg(test)] Pure code motion: moved helpers are bumped to pub(super); the public API is unchanged (lib.rs still re-exports generate_operator_manifest and the other entrypoints). The file drops off the hk.pkl max-lines exclude now that every module is under the 2000-line cap.
Break the 2,262-line generator.rs into a generator/ directory so each concern can be read and reviewed in isolation: - mod.rs: the template constants, the RegistrationMode / CloudFormationOptions / CloudFormationTarget types, the generate_cloudformation_template entrypoint, YAML serialization, stack validation, and the shared resource/output helpers - parameters.rs: the standard/network/compute parameter blocks, the supported-region and custom-domain rules, the standard conditions, and the parameter-default helpers - expressions.rs: the kubernetes/network/domains/management settings expressions, the JSON <-> CfExpression bridge, and the CfParameter / CfOutput constructors - tests.rs: the unit test module, still gated by #[cfg(test)] Pure code motion: moved helpers are bumped to pub(super); the public API is unchanged (lib.rs still re-exports generate_cloudformation_template and the other entrypoints). The file drops off the hk.pkl max-lines exclude now that every module is under the 2000-line cap.
The EC2 client lived in a single 3616-line file. Split it into an ec2/ module mirroring the existing client layout: api (the Ec2Api trait), client (Ec2Client and its impls plus the cpu-options unit tests), and types/ grouped by resource (common, network, security_group, instance, volume, launch_template). Pure code motion with import redistribution and module wiring; no behavior change. Remove the file from the max-lines exclude list now that every resulting file is under the 2000-line cap.
Break the 3,140-line gcp_compute_client_tests.rs into a directory test target so each area can be read and reviewed in isolation. The binary name (gcp_compute_client_tests) and all 17 tests are unchanged: - main.rs: crate doc, module wiring, and the framework smoke test - context.rs: the ComputeTestContext fixture and its cleanup/wait helpers - vpc.rs, load_balancing.rs, disks.rs, instances.rs, ssl_proxy.rs: the comprehensive lifecycle tests, one area per module - errors.rs: the not-found/error-mapping tests plus the operation-status test Pure code motion: no test logic changed. Fixture items used across modules are bumped from private to pub(crate), per-module imports are consolidated, the mid-function rcgen import is hoisted to the top of ssl_proxy.rs, and a mislabeled "Error Handling Tests for New APIs" banner that sat above the SSL lifecycle test is dropped. The file is removed from the hk.pkl max-lines exclude list now that every module is under the 2,000-line cap.
Break the 2,265-line aws_s3_client_tests.rs into a directory test target so each area can be read and reviewed in isolation. The binary name (aws_s3_client_tests) and all 37 tests are unchanged: - main.rs: the crate attribute and module wiring - context.rs: the S3TestContext fixture, its cleanup helpers, and the put_test_object helper - bucket.rs: bucket lifecycle, policy, lifecycle-config, public-access block, name validation, and notification-config tests - object.rs: object put/get/head, listing, delete, and empty-bucket tests - versioning.rs: bucket-versioning and versioned-object tests - location.rs: the get_bucket_location tests Pure code motion: no test logic changed. Fixture items used across modules are bumped from private to pub(crate) and per-module imports are consolidated. The file is removed from the hk.pkl max-lines exclude list now that every module is under the 2,000-line cap.
Break the 4,726-line distribution.rs into a distribution/ directory so each concern can be read and reviewed in isolation: - mod.rs: DistributionArtifactCleanup, setup_distribution and prepare_distribution, the flow-availability and stack-settings helpers, the cross-domain flow orchestrators (run_cloudformation_k8s, run_terraform_cloud/k8s, run_onprem_k8s), and stack import/finalize - cleanup.rs: post-error cleanup and retained-resource teardown - cloudformation.rs: run_cloudformation_aws and the CloudFormation request/output/stack-event helpers - terraform.rs: the Terraform apply/import flow, tfvars, outputs, and handoff-debug helpers - helm.rs: chart/values rendering, the Kubernetes helm target and kubeconfig materialization, and image-rewrite helpers - permissions.rs: the GCP and Azure management permission probes - env.rs: cloud env assembly (aws/gcp/azure/terraform) - exec.rs: the process runners and shell-quote helpers - tests.rs: the unit test module, still gated by #[cfg(test)] Pure code motion: moved helpers are bumped to pub(super); the public API is unchanged (setup_distribution and DistributionArtifactCleanup are still defined in mod.rs). The file drops off the hk.pkl max-lines exclude now that every module is under the 2,000-line cap.
Break the 2,700-line heartbeat.rs into a heartbeat/ directory so each family of provider heartbeat types can be read and reviewed on its own: - mod.rs: the ResourceHeartbeat envelope, ObservedInventoryBatch / ObservedResourceSample, shared health/lifecycle/backend/collection enums, event snapshots, raw snippets, and metric types - storage.rs: StorageHeartbeatData and the S3/Blob/GCS/Local variants - workload.rs: worker, container, and daemon heartbeat data plus the runtime unit and pod status types - cluster.rs: compute-cluster and kubernetes-cluster heartbeat data with capacity/drain/fleet and node status types - data_services.rs: queue, kv, postgres, and vault families - platform.rs: service-account, network, and remote-stack-management families - registry_build.rs: artifact-registry, build, and service-activation families - azure_platform.rs: Azure resource-group, storage-account, container apps environment, and service-bus-namespace families - tests.rs: the serde round-trip unit module, still gated by #[cfg(test)] Pure code motion: submodules are glob re-exported from mod.rs so the public API at the crate root is unchanged. Drop the file from the hk.pkl max-lines exclude list now that every module is under the cap.
The generator split dedented the tests module by one level to extract it into its own file, which unintentionally stripped four spaces from a line inside a raw-string YAML literal, placing deploymentLabelValue at the document root instead of under logCollector.scope. Helm 4 schema validation rejects the root-level key, failing log_collector_enabled_chart_lints_and_templates. Restore the four-space indentation so the literal is byte-identical to pre-split. Verified with helm 4.2.3 + kubeconform: alien-helm 15/15 tests pass.
Relocate post-split changes from main into their focused modules and extract test modules that crossed the 2,000-line limit after the merge.
…files # Conflicts: # crates/alien-terraform/src/generator.rs # crates/alien-test/src/distribution.rs
…ings' into refactor/split-large-files
…ings' into refactor/split-large-files # Conflicts: # crates/alien-infra/src/worker/azure/mod.rs
# Conflicts: # crates/alien-test/src/distribution.rs
…ings' into refactor/split-large-files # Conflicts: # crates/alien-cloudformation/src/generator.rs # crates/alien-terraform/src/generator.rs
Summary
Splits oversized Rust and TypeScript source files into focused modules and adds one canonical source-length guard shared by local hooks and CI.
client-sdks/**andpackages/sdk/src/worker-runtime/generated/**.The largest changed source file is 987 lines.
crates/alien-terraform/src/generator/mod.rsis 833 lines after extractinggenerator/dependencies.rs.Main splits
up, heartbeat, instance catalog, build, distribution, and registry-proxy modulesReview evidence
scripts/check-max-lines.shpasses and is invoked by both Fast/Fork CI and local hooks.rustfmt --checkandgit diff --checkpass.cargo check -p alien-terraform --all-targetspasses.cfn-lint: 1/1.fmt + init + validate: 3/3 using Terraform 1.12.2.f64cc8a8c8102e8ebee6f09362ad40e45c781e8b.Exact review range
69568024fa3dc3a2d82e92897b90c89533f3dd0af64cc8a8c8102e8ebee6f09362ad40e45c781e8bReview move-heavy changes with:
Merge order
main.